Tracking performance of the RLS algorithm applied to an antenna array in a realistic fading environment

In this paper, frequency domain techniques are used to derive the tracking properties of the recursive least squares (RLS) algorithm applied to an adaptive antenna array in a mobile fading environment, expanding the use of such frequency domain approaches for nonstationary RLS tracking to the interference canceling problem that characterizes the use of antenna arrays in mobile wireless communications. The analysis focuses on the effect of the exponential weighting of the correlation estimation filter and its effect on the estimations of the time variant autocorrelation matrix and cross-correlation vector. Specifically, the case of a flat Rayleigh fading desired signal applied to an array in the presence of static interferers is considered with an AR2 fading process approximating the Jakes' fading model. The result is a mean square error (MSE) performance metric parameterized by the fading bandwidth and the RLS exponential weighting factor, allowing optimal parameter selection. The analytic results are verified and demonstrated with a simulation example.